Video coding apparatus according to a feature of a video picture

ABSTRACT

A variance between sequential video pictures is extracted, and then, a GOP boundary position is decided based on inter-frame variance information. Furthermore, simple motion estimation is carried out with respect to video pictures inside one GOP. If a motion variation between the video pictures is large, a small predictive frame interval is taken; to the contrary, if the motion variation is small, a large predictive frame interval is taken. The simple motion estimation is carried out between two downscaled feature planes at a timewise fixed interval with respect to a video picture which is discriminated to be an interlaced video picture, wherein a motion compensatory prediction error at that time is output as image variance information. If the image variance is small, coding is conducted by a frame structure; to the contrary, if the image variance is large, the coding is conducted by a field structure. With the above-described processing, it is possible to provide a video coding apparatus for deciding a GOP size and the predictive frame interval according to the feature of the input video picture, and another video coding apparatus for adaptively switching the coding by the frame/field structures according to the feature of the input video picture.

This application is a divisional application of prior application Ser.No. 09/515,896 filed Feb. 29, 2000.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a video coding apparatus and, moreparticularly, to a video coding apparatus for performing coding by usingmotion compensatory prediction of a digital video signal.

2. Description of the Related Art

Among highly efficient coding systems for coding sequentially inputvideo signals in a fewer code quantity, coding systems by the use of themotion and correlation between video pictures of video signals includemotion compensatory prediction coding which decodes to reproduce a videopicture coded in the past and uses motion information per small blockderived from the video picture. One example of the conventional motioncompensatory prediction coding is illustrated in FIG. 1.

In FIG. 1, when an input video signal 1 of a first screen is input, eachof switches is controlled to be connected onto a side (1) by a codingmode control section 12, and the input video signal 1 is input directlyinto an orthogonal transform unit 3 in order to achieve high codingefficiency. The input video signal 1 is orthogonally transformed byusing DCT (discrete cosine transform) or the like in the orthogonaltransform unit 3. An orthogonal transform coefficient is quantized by aquantizer 4. The resultant quantization coefficient is converted into avariable length code such as a Huffman code by a first variable lengthencoder 5, and then, is input into a video duplexer 15.

In the meantime, the quantization coefficient input into an inversequantizer 6 is inversely quantized, and then, video picture data isrestored by an inverse orthogonal transform unit 7. The restored videopicture data is stored in a frame memory 9. Moreover, coded data 13transmitted from the first variable length encoder 5 and quantizationinformation 18 transmitted from the quantizer 4 is duplexed by the videoduplexer 15, to be output as a coded video data output 16.

When another input video signal 1 of a next screen is input, each of theswitches is controlled to be connected to a contact on a side (2) by thecoding mode control section 12, so that the input video signal 1 isinput into a predictive signal subtraction section 2 and a motioncompensator 10. In the motion compensator 10, a motion vector isdetected based on the input video signal 1 and a reference video pictureinput from the frame memory 9, and then, is input into a positionshifter 11 and a second variable length encoder 14. In the secondvariable length encoder 14, motion vector information is converted intoa variable length code such as a Huffman code, thus to be input into thevideo duplexer 15.

In the position shifter 11, a video signal designated by the motionvector is extracted from the frame memory 9, and thereafter, is outputas a motion compensatory predictive signal to the predictive signalsubtraction section 2 and a local decoding addition section 8. In thepredictive signal subtraction section 2, the motion compensatorypredictive signal is subtracted from the input video signal 1, so that aprediction error thereof is coded. A prediction error signal isorthogonally transformed by using DCT (discrete cosine transform) or thelike in the orthogonal transform unit 3 in order to achieve high codingefficiency. The signal quantized by the quantizer 4 is converted into avariable length code such as a Huffman code in the first variable lengthencoder 5. In order to use the same predictive signal as that on adecoding side, the quantization coefficient obtained by the quantizer 4is inversely quantized by the inverse quantizer 6, and then, theprediction error signal is locally decoded by the inversely orthogonaltransform unit 7. Furthermore, the motion compensatory predictive signalis added with the prediction error signal decoded by the local decodingaddition section 8, and then, is stored in the frame memory 9.

In view of convenience of highly efficient coding and decodingreproduction, the video picture is coded by combining three kinds ofvideo coding systems for P, B and I frames.

A minimum unit of video pictures, which are formed by combining thethree kinds of video coding systems and can be decoded independently ofeach other, is referred to as “a GOP (a Group of Pictures)”. Thecombination of the coding systems is referred to as “a GOP structure”. Aframe first coded inside one GOP is intra-frame coding (an I frame).FIG. 2 illustrates an example of a GOP. In FIG. 2, the number of framesincluded in one GOP is referred to as a GOP size, and an intervalbetween P frames or between an I frame and a P frame is referred to as apredictive frame interval.

An I frame inserting interval has been conventionally constantirrespective of the feature of the input video picture: namely, the GOPsize has been fixed, so that intra-frame coding has been forciblycarried out per predetermined number of frames. Consequently, the Iframe has been inserted even in the case where the input video picturehas the high correlation with the reference video picture and codingefficiency can be enhanced by using inter-frame prediction coding.

As for the predictive frame interval, a predictive frame interval ofhighest coding efficiency depends on the feature of the video picture.For example, a video picture of a swift motion can be predicted from thereference video picture with high efficiency by shortening thepredictive frame interval, thus enhancing the coding efficiency. To thecontrary, in the case of little variation, the predictive frame intervalis prolonged, thereby enhancing the coding efficiency. However, sincethe predictive frame interval is fixed to about 0.1 secondirrespectively of the feature of the video picture in the conventionalsystem, the coding efficiency can not be enhanced.

Furthermore, in a video picture compression system capable of coding byeither a frame structure or a field structure, there can be used eithercoding by “the field structure” in which one video picture to be codedis coded in a manner corresponding to one field video picture or codingby “the frame structure” in which one video picture to be coded is codedin a manner corresponding to one interlaced frame video picture.However, in the prior art, it is previously designated from the outsideas to which is selected out of the frame structure and the fieldstructure before the video picture is coded, so that the video pictureto be input is coded by fixedly using the designated structure, therebyoutputting coded data. That is, the coding is carried out by the fixedpicture structure irrespective of the feature of the video picture.

Therefore, even in the case of coding a video picture of a swift motionin which the coding efficiency can be enhanced by adopting the fieldstructure, the coding by the frame structure is continued if the framestructure is previously designated as the coding picture structure,resulting in degradation of the coding efficiency. To the contrary, inthe case where the coding by the field structure is previouslydesignated, the coding efficiency cannot be enhanced since the fieldstructure is fixedly used even if the coding efficiency can be enhancedby the frame structure.

Additionally, in the case where it is not found whether the input videopicture is an interlaced video picture or a non-interlaced videopicture, high coding efficiency can be achieved by a 2-step system inwhich it is previously discriminated by some method whether or not theinput video picture is an interlaced video picture, and thereafter, thepicture structure is switched from the outside at the time of codingbased on the discrimination information. Such a 2-step system isunavailable on the assumption of coding in real time.

SUMMARY OF THE INVENTION

The present invention has been accomplished in an attempt to solve theabove problems experienced by the prior art. Therefore, an object of thepresent invention is to provide a video coding apparatus in which codingefficiency can be enhanced and a quality of a coded video picture can bestabilized by adaptively changing a GOP size and a predictive frameinterval according to the feature of an input video picture orvariations of the feature of the input video picture.

Another object of the present invention is to provide a video codingapparatus in which coding efficiency can be enhanced and a quality of acoded video picture can be stabilized by automatically discriminatingwhether an input video picture having no information on the feature orstructure of a video picture is an interlaced input video picture or anon-interlaced input video picture and analyzing the feature of thevideo picture to be input, so as to adaptively change a picturestructure in video picture compressing/coding to a frame structure or afield structure.

In order to achieve the above objects, the present invention has a meansfor detecting a variance between the video pictures based on informationon sequentially input video pictures, determining the correlationbetween the video pictures based on the detected information, anddeciding the video picture for which an intra-frame coding system isused according to the degree of the correlation.

With this characteristic, a GOP size depends on the feature of the videopicture.

Furthermore, the present invention has a means for detecting a motionfeature between the input video pictures so as to decide an optimumpredictive frame interval.

With this characteristic, the optimum predictive frame interval can bedecided based on the motion feature between the input video pictures.

Moreover, the present invention has a means for discriminating whethereach of sequentially input video pictures is an interlaced video pictureor a non interlaced video picture, wherein coding by the field structureis selected if the video picture is an interlaced video picture whilecoding by the frame structure is selected unless the video picture is aninterlaced video picture.

Additionally, the present invention calculates a variance of a videopicture based on an interlaced video picture to be input so as to switchcoding by the frame/field structures based on the calculation value.

With these features, it is possible to prevent any degradation of thecoding efficiency caused by a variation in feature of the input videopicture, which was inevitable at the time of fixed selection of theframe/field structures in the prior art. Furthermore, since thediscrimination as to whether the input video picture is an interlacedVideo picture or a non-interlaced video picture, which need bedetermined before the coding, is automatically detected at the time ofthe coding, the efficient coding can be carried out irrespective of thefeature or structure of the input video picture.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a conventional motioncompensatory prediction coding apparatus, to which the present inventionis applied.

FIG. 2 is a view illustrating a conventional GOP structure.

FIG. 3 is a block diagram illustrating a motion compensatory predictioncoding apparatus encompassing the present invention.

FIG. 4 is a block diagram illustrating the configuration of a GOPstructure deciding section in a first preferred embodiment according tothe present invention.

FIGS. 5A and 5B are explanatory views of a method for calculating avariance between two pixels.

FIG. 6 is a view explanatory of creation of a downscaled video picturefor the purpose of simple motion estimation.

FIG. 7 is a block diagram illustrating a second preferred embodimentaccording to the present invention.

FIG. 8 is a block diagram illustrating a third preferred embodimentaccording to the present invention.

FIG. 9 is a block diagram illustrating a fourth preferred embodimentaccording to the present invention.

FIG. 10 is a graph illustrating simulation results according to thepresent invention.

FIG. 11 is a block diagram illustrating the configuration in a fifthpreferred embodiment according to the present invention.

FIG. 12 is a view illustrating the configuration of a frame videopicture.

FIG. 13 is a view explanatory of a pixel for calculating an absolutedifference.

FIG. 14 is a view explanatory of creation of a downscaled feature plane.

FIG. 15 is a view explanatory of processing of the simple motionestimation.

FIG. 16 is a block diagram illustrating the configuration in a sixthpreferred embodiment according to the present invention.

FIG. 17 is a block diagram illustrating the configuration in a seventhpreferred embodiment according to the present invention.

FIG. 18 is a block diagram illustrating the configuration in an eighthpreferred embodiment according to the present invention.

FIG. 19 a block diagram illustrating the configuration in a ninthpreferred embodiment according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention will be described in detail below in reference tothe drawings. FIG. 3 is a block diagram illustrating the configurationin a first preferred embodiment according to the present invention.Although the coding apparatus illustrated in FIG. 1 is used as a videopicture coding system in the description below, the present invention isnot limited to such a coding apparatus. The same reference numerals asthose used in FIG. 1 denote like or corresponding constituent elements.

This preferred embodiment is characterized in that the features ofsequentially input video signals are analyzed based on the videosignals, so that a GOP structure is decided according to the features,thereby performing coding processing based on the GOP structure.

In FIG. 3, first, the features of sequentially input video signals areanalyzed in a GOP structure decision section 20, which then decides aGOP structure according to the input video picture based on thefeatures. Subsequently, when the video picture is coded, a GOP structureinformation signal 21 is output to a coding mode control section 12; inthe meanwhile, coding complexity prediction information 22 is output toa coding bit rate control section 17. Operation other than theabove-described operation is similar to that of the coding apparatusillustrated in FIG. 1, and so, its description will be omitted.

FIGS. 4, 7, 8 and 9 are block diagrams illustrating preferredprocessings of the GOP structure decision section 20 illustrated in FIG.3. First of all, the processing will be explained in reference to FIG. 4illustrating the first preferred embodiment according to the presentinvention. First, a frame memory 31 stores therein the sequentiallyinput video signals. The frame memory 31 can store therein videopictures equivalent or more to the maximum GOP size.

An inter-frame variance analysis section 32 calculates a variance of atarget video picture based on the video pictures stored in the framememory 31 and a timewise immediately preceding video picture adjacent tothe target video picture, and then, outputs inter-frame varianceinformation A resulting from the calculation to a GOP boundary positiondecision section 33. Here, although the target video picture and theimmediately preceding video picture are used for the calculation of theinter-frame variance information A, video pictures other than theimmediately preceding video picture may be used.

The GOP boundary position decision section 33 decides a position optimumfor a GOP boundary inside the frame memory 31 based on the inter-framevariance information A output from the inter-frame variance analysissection 32, and then, outputs the decided position as GOP boundaryposition information B. Upon this decision of the GOP boundary position,the video pictures prior to the decided GOP boundary position storedinside the frame memory 31 constitute one GOP.

A simple motion estimation section 34 decides a reference video pictureout of the video pictures equivalent to one GOP size stored in the framememory 31 after an I frame inserting position, i.e., after the decisionof the size of one GOP by the GOP boundary position decision section 33,and then, outputs motion feature prediction information C by simplemotion estimation between the reference video picture and the othervideo picture.

Subsequently, a predictive frame interval decision section 35 decides apredictive frame interval based on the motion feature predictioninformation C input from the simple motion estimation section 34, andthen, outputs predictive frame interval information D.

The inter-frame variance information A, the GOP boundary positioninformation B, the motion feature prediction information C and thepredictive frame interval information D are input into a codingcomplexity prediction section 37, which predicts coding complexity ateach of I, P and B frame coding modes so as to output the resultantinformation as coding complexity prediction information E to a codingbit rate control section 17.

The coding bit rate control section 17 controls a coding bit rate incoding the input video picture in consideration of the coding complexityprediction information E input from the coding complexity predictionsection 37. The GOP boundary position information B and the predictiveframe interval information D are output also to a coding mode controlsection 12, which controls switches at the time of the coding in the GOPstructure decided on the basis of the information B and D.

After the decision of the structure of one GOP inside the frame memory31, the frame memory 31 outputs a video signal to the predictive signalsubtraction section 2 shown in FIG. 3 in order to code each videopicture of the GOP. Information on the output video signal is erasedfrom the frame memory 31.

Upon completion of the coding of one GOP, the frame memory 31 storestherein video pictures input in sequence posterior to the residual videopictures stored therein. When the frame memory 31 stores therein thevideo signals equivalent to the GOP size to the maximum, it performs theprocessing of deciding a next GOP structure. This processing isrepeated.

Next, a description will be given in detail of the operation of each ofthe constituent elements illustrated in FIG. 4.

Next, description will be given in detail of the operation of each ofthe constituent elements illustrated in FIG. 4.

First, the frame memory 31 stores therein the sequentially input videosignals. The number of video pictures to be stored is equivalent or moreto the maximum GOP size which is decided at the time of the coding. Theframe memory 31 outputs the video signals to the inter-frame varianceanalysis section 32 and the simple motion estimation section 34,respectively. When the structure of one GOP is decided in the storedvideo pictures, the video signal is output to the video codingapparatus. Consequently, the output video signal is erased from theframe memory 31, and then, a newly input video signal is stored in thatvacant region in the frame memory 31.

Subsequently, the inter-frame variance analysis section 32 fetches twopieces of video picture information from the frame memory 31, tocalculate the inter-frame variance information A. The calculatingmethods include a method for calculating a variance based on theintra-frame sum of absolute differences of pixel information on the twovideo pictures at the same position; and a method for dividing the videopicture into small blocks, determining dispersion values of pixels inthe small blocks, and calculating the intra-frame sum of absolutedifferences between frames in which the dispersion values arerepresentative of the small blocks.

In the former deciding method, as shown in, for example, FIG. 5A,assuming that pixel values of the video pictures (i) and (j) aredesignated by Pi1, Pi2, . . . , Pin and Pj1, Pj2, . . . , Pjn,respectively, the intra-frame sum A of the absolute differences isexpressed by the following equation (1):

$\begin{matrix}{A = {\sum\limits_{k = 1}^{n}\;{{{Pik} - {Pjk}}}}} & (1)\end{matrix}$

Furthermore, in the latter deciding method, as shown in, for example,FIG. 5B, assuming that dispersion values of the small blocks in thevideo pictures (i) and (j) are designated by σi1, σi2, . . . , σim andσj1, σj2, . . . , σjm, respectively, the intra-frame sum A of theabsolute differences is expressed by the following equation (2):

$\begin{matrix}{A = {\sum\limits_{k = 1}^{m}\;{{{\sigma\;{ik}} - {\sigma\;{jk}}}}}} & (2)\end{matrix}$

Although each of the pixel values in the decision methods is processedby using only luminance, it may be processed by using chrominance orusing both-luminance and chrominance. The inter-frame varianceinformation A calculated by the inter-frame variance analysis section 32is output to the GOP boundary position decision section 33 and thecoding complexity prediction section 37.

The GOP boundary position decision section 33 decides a video pictureimmediately before the frame as the GOP boundary based on theinter-frame variance information A input from the inter-frame varianceanalysis section 32 in the case where the value of the information Aexceeds a predetermined threshold value. Otherwise, the GOP boundaryposition decision section 33 may decide a video picture immediatelybefore a video picture having a maximum value of the information A asthe GOP boundary based on the inter-frame variance information A on allof the video pictures stored inside the frame memory 31; or it maydecide it based on a logical sum or a logical product obtained by boththe system using the threshold value and the system using the maximumvalue. The GOP boundary position information B obtained in the GOPboundary position decision section 33 is output to the simple motionestimation section 34, the coding mode control section 12 and the codingcomplexity prediction section 37.

After one GOP size with respect to the video pictures inside the framememory 31 is decided in the GOP boundary position decision section 33,the simple motion estimation section 34 performs simple motionestimation processing in order to predict motion information on thevideo picture inside the GOP. In a method for collecting most accuratemotion information, the input video picture is divided into small blockseach composed of 8×8 pixels or 16×16 pixels, each of the small blocks issubjected to motion estimation, and consequently, the most accuratemotion information is determined based on the resultant motioninformation on each of the small blocks in the same manner as the motionestimation processing by the motion compensator 10 in the video codingapparatus for the video pictures illustrated in FIG. 3. However, since aprocessing quantity required for the motion estimation is huge, additionprocessing of 2³¹ times or more is required for the motion estimationprocessing of one video picture in the case where, for example, thevideo picture size is 720×480 pixels and the motion estimation fallswithin the range of ±16 pixels. As a consequence, the present inventionuses the means for predicting the motion information on the videopicture based on the information resulting from the simple motionestimation processing of a small processing quantity performed in thesimple motion estimation section 34. Description will be given below ofthe simple motion estimation processing.

First, one video picture decided inside the GOP is selected as areference video picture. Thereafter, the reference video picture isdivided into small blocks. Subsequently, a downscaled video picture, inwhich the small block is expressed by one representative value, iscreated. Here, the dispersion of all of the pixel values inside thesmall block, for example, can be used for calculation of therepresentative value. The oldest video picture out of the target GOP isselected as the reference video picture, but other video pictures may beselected.

Next, in order to grasp the motion features in comparison with thereference video picture, the target video pictures are determined, andthen, the downscaled video pictures of these video pictures are created.Thereafter, the motion estimation processing is performed by the use ofthe downscaled video pictures of both of the reference video picture andthe target video picture. Although according to the present invention,the motion estimation processing is performed with respect to all of thevideo pictures except the reference video picture inside the GOP, notall of the video pictures but some selected video pictures may besubjected to the motion estimation processing.

FIG. 6 illustrates a method for creating the downscaled video picture.Assuming that a video picture to be input is composed of M pixels in ahorizontal direction multiplied by N pixels in a vertical direction anda small block is composed of 8 pixels in the horizontal and verticaldirections, respectively, a representative value of the small block isone with respect to 64 pixels, so that a downscaled video picture to becreated is composed of M/8 in the horizontal direction multiplied by N/8in the vertical direction in the case where N and M each are a multipleof 8. Furthermore, the size of the small block maybe processed not inthe size of 8×8 pixels, but in the size of 16×16 pixels or in the sizesof all other rectangular blocks.

Although the dispersion value of each of the pixel values inside thesmall block is used for the calculation of the representative value persmall block in this system, an average value, a standard deviation, anabsolute error sum with respect to the average value or combinationsthereof may be used. Although a luminance is used herein as the pixelvalue, a luminance and/or a chrominance may be used.

The motion estimation is generally performed per small block, so that avector indicating a position of a smallest difference is calculated,thereby obtaining the motion feature. However, accuracy of the motionvector information is low since the downscaled video picture is used inthis system. Consequently, the smallest motion compensatory predictionerror at the time of the motion estimation based on the downscaled videopicture information is a motion feature value of the video picture,i.e., the motion feature prediction information C as an index of themagnitude of the motion of the entire video picture. A square error, anabsolute error, an absolute error at a square root may be used for thecalculation of the motion compensatory prediction error.

The obtained motion compensatory prediction error as the motion featureprediction information C is input into the predictive frame intervaldecision section 35, which decides a predictive frame interval based onthe motion feature prediction information C. The predictive frameinterval is small in the case where a motion or a variation between thevideo pictures is great in coding the video pictures; to the contrary,the predictive frame interval is large in the case where a motion or avariation between the video pictures is small in coding the videopictures, thereby achieving most efficient coding. Consequently, inorder to grasp the motion feature over one GOP, the motion featureprediction information C on all of the reference video picture and othervideo pictures inside one GOP is obtained, and then, the average valuethereof is obtained. The average value is used as a representativevalue, on the basis of which the predictive frame interval isdetermined. One of the characteristics of the present invention residesin that the inversely proportional relationship is established betweenthe predictive frame interval and the obtained average value. Besidesthe method for using the average value, a maximum value or a minimumvalue may be used for the calculation of the representative value insideone GOP.

Since a relative motion quantity with respect to a pixel becomes largein the case where the resolution of the video picture to be input ishigh, the inversely proportional relationship is established between theresolution of the video picture and an optimum predictive frameinterval. Another characteristic of the present invention resides inthat the inversely proportional relationship with respect to theresolution information on the video picture is considered in decidingthe predictive frame interval. The decided predictive frame intervalinformation D is output to the coding complexity prediction section 37and the coding mode control section 12. The GOP boundary positioninformation B and the predictive frame interval information D aretransmitted together to the coding mode control section 12, in which theswitches are controlled based on the information B and D in coding thevideo pictures.

The inter-frame variance information A, the GOP boundary positioninformation B, the motion feature prediction information C and thepredictive frame interval information D are input into the codingcomplexity prediction section 37, which calculates coding complexityprediction information E as an index of generated code quantityprediction in coding at a coding mode of each of the I, P and B frames,and then, outputs the coding complexity prediction information E to thecoding bit rate control section 17.

When the processing proceeds to coding of a new GOP, the coding bit ratecontrol section 17 renews coding complexity prediction information ateach of the coding modes based on the coding complexity predictioninformation E input from the coding complexity prediction section 37.Coding complexity prediction information used at past frames having thesame coding mode has been conventionally used irrespectively ofswitching of the input video picture or fluctuations. Consequently, inthe case where an input video picture has suffered from a large changesuch as a change in scene, the video picture has been influenced bycoding complexity prediction information on a frame having nocorrelation, with an attendant problem of markedly degradation of aquality of the video picture. However, since the prediction is carriedout based on the information of the video picture to be coded accordingto the present invention, the above-described problem can be solved.

Subsequently, explanation will be made on a method for calculating thecoding complexity prediction information E at each of the coding modes.A video picture which is coded as an I frame is divided into smallblocks and the dispersion of a pixel value per small block is decided,so that the coding complexity prediction information E at the I frame iscalculated by a product of an intra-frame average of the dispersionsmultiplied by a fixed value SI as a scaling parameter. Luminanceinformation and/or chrominance information may be used as the pixelvalue.

In the case where an absolute difference calculated between thedispersion of the pixel value per small block and dispersion of a pixelvalue of an adjacent small block exceeds a threshold value, it is judgedthat the small block region of the input video picture includes edgeinformation such as an outline, so that the coding bit rate controlsection 17 takes the judgement into consideration so as to assign manycoding quantities in coding the small block region.

An average of the motion compensatory prediction errors is obtainedbased on all of the motion feature prediction information C inside thetarget GOP, and then, the coding complexity prediction information E atthe P frame is calculated by a product of the average value multipliedby a fixed value SP as a scaling parameter. Otherwise, the codingcomplexity prediction information E at the I frame may be scaled for thecalculation.

The coding complexity prediction information E at the B frame iscalculated by a product of the coding complexity prediction informationat the P frame multiplied by a fixed value SB as a scaling parameter.

Subsequently, a second embodiment according to the present invention isillustrated in FIG. 7. The present embodiment is configured such thatthe processing of the coding complexity prediction section 37 in thefirst embodiment illustrated in FIG. 4 is omitted.

Next, a third embodiment according to the present invention isillustrated in FIG. 8. The present embodiment is configured such thatthe processing concerned in the decision of the GOP size in the secondembodiment illustrated in FIG. 7 is omitted. In the present embodiment,a GOP size is fixed with a length designated in advance. In each GOP, anoptimum predictive frame interval is adaptively varied based on motionfeature prediction information C.

Subsequently, a fourth embodiment according to the present invention isillustrated in FIG. 9. The present embodiment is configured such thatthe processing concerned in the decision of the predictive frameinterval in the second embodiment illustrated in FIG. 7 is omitted. Inthe present embodiment, the predictive frame interval is fixedlydesignated in advance. Only a GOP size is adaptively varied based oninter-frame variance information A which is a feature of an input videopicture.

As is obvious from the above description, since according to the presentinvention the GOP size is decided according to the feature or variationof the input video picture, the GOP size can be decided in a manneradaptive to the variation of the input video picture. Therefore, it ispossible to avoid degradation of the coding efficiency or fluctuation ofa quality of the video picture which may occur in the case of codingwith the fixed GOP size.

Moreover, since the motion feature of the video picture inside the GOPcan be detected based on the decided GOP size and the predictive frameinterval according to the motion feature can be set, the predictiveframe interval can be taken according to the motion feature of the inputvideo picture. Consequently, it is possible to enhance the codingefficiency more than the case of the conventional coding at the fixedpredictive frame interval.

Additionally, the coding complexity prediction information used in thepreceding GOP has been considered even in the case where there has beenno correlation in video picture feature between a preceding GOP and atarget GOP due to a scene change or the like in the prior art, therebyinducing markedly fluctuations or deterioration of a quality of thevideo picture to be coded or degradation of the coding efficiency. Incontrast, according to the present invention, the coding complexityprediction information is calculated based on the features of the videopictures inside the target GOP after the completion of the coding of oneGOP and before coding of a next GOP, so that a video picture can becoded with a stable quality without any influence of the feature of thevideo picture inside an irrelevant GOP.

FIG. 10 shows the simulation result on a video picture with a change inscene in the MPEG2 system. In this simulation, under the condition wherecompression coding was carried out at a coding rate of 4 Mbit/s,fluctuation of PSNR was small and the quality of the video picture couldbe improved by 0.65 dB according to the present invention in comparisonwith the coding in the prior art in which the GOP size was fixed to 15frames and the predictive frame interval was fixed to 3 frames.

Next, a fifth embodiment according to the present invention will beexplained in reference to FIG. 11. In the present embodiment, it isdiscriminated based on each of sequentially input video signals(stationary video signals) whether or not an input video picture is aninterlaced video picture. If the input video picture is an interlacedvideo picture, a downscaled feature plane is created, and then, codingin a frame/field structure is decided based on the result of simplemotion estimation processing by the use of the downscaled feature plane.

In FIG. 11, an interlaced/non-interlaced video discriminant section 51discriminates whether each of sequentially input video signals 1 is aninterlaced video signal or a non-interlaced video signal. Thediscrimination result is output as interlaced/non-interlaceddiscriminant information 52 to a downscaled feature plane creationsection 53. The downscaled feature plane creation section 53 createsdownscaled feature plane information 54 in consideration of the featureof the video picture with respect to the video picture which isdiscriminated as the interlaced video picture in theinterlaced/non-interlaced video discriminant section 51, and outputs thedownscaled feature plane information 54 to a simple motion estimationsection 55. The simple motion estimation section 55 performs simplemotion estimation processing between two downscaled feature planes, andoutputs the resultant motion compensatory prediction error as imagevariance information 56 to a frame/field structure decision section 57.

Based on the image variance information 56 obtained by the simple motionestimation section 55, the frame/field structure decision section 57decides coding by the frame structure in the case of a small variancewhile coding by the field structure in the case of a large variance, andoutputs the result as picture structure information 58 to a video codingsection 59. The video coding section 59 performs video coding withrespect to the input video signal 1 in response to picture structureinformation 58 indicated by the frame/field structure decision section57, and outputs coded data 16. Here, the video coding section 59switches the operations of, for example, the motion compensator 10, thefirst variable length encoder 5 and the second variable length encoder14, illustrated in FIG. 1, to the operation adaptive to coding in theframe/field structure according to the designation of the frame/fieldstructure based on the picture structure information 58.

Next, description will be given of one example of the configuration andoperation of each of the constituent elements in FIG. 11. First,explanation will be made on the interlaced/non-interlaced videodiscriminant section 51. The discrimination as to whether or not thevideo picture is an interlaced video picture is decided by thecalculation with some adjacent pixels based on the video pictureinformation to be input. FIG. 12 illustrates the configuration of framevideo information to be input. The video information is composed of thearray of space wise uniformly arranged pixels. Based on the videoinformation, five pixel values continuous in a vertical direction at anarbitrary position are taken, and then, an absolute difference betweentwo pixels is calculated, as illustrated in FIG. 13.

There are calculated absolute differences between pixels belonging tothe same fields of 0 and −2, 0 and 2, and −1 and 1 in five pixels p(−2)to p(2), wherein a pixel positioned at the center in the verticaldirection is designated by p(0), and absolute differences between pixelsbelonging to different fields of 0 and −1, and 0 and 1. It is verifiedwhether or not the condition expressed by inequality (3) below issatisfied:Max(d(0,−2),d(0,2),d(−1,1))<threshold value  (3)

Subsequently, if the condition expressed by inequality (3) above issatisfied, it is further verified whether or not the condition expressedby inequality (4) below is satisfied:(Max(d(0,−2),d(0,2),d(−1,1))+offset)<Min(d(0,−1),d(0,1))  (4)

Here, d(a,b) represents an absolute difference between a and b;Max(a,b,c), a maximum value of a, b and c; and Min(a,b,c), a minimumvalue of a, b and c. That is, in the case where the pixel valuesbelonging to the same field are similar to each other and the maximumabsolute difference is less than the threshold value (a fixed value), itis verified whether or not the minimum value of the absolute differencesat the different fields exceeds a value obtained by adding an offset (afixed value) to the maximum absolute difference at the same field. Thisprocessing is performed with respect to all of the pixels or thearbitrary number of positions inside the video picture. In the casewhere the points satisfying inequalities (3) and (4) exceed apredetermined rate of the points satisfying inequality (3), the videopicture is discriminated as an interlaced video picture, and then, theresult is output as the interlaced/non-interlaced discriminantinformation 52 per frame to the downscaled feature plane creationsection 53.

Furthermore, although the description has been given of the example inwhich the discrimination of the interlaced/non-interlaced video pictureis performed by the use of the five pixels in the vertical direction,the number of pixels required for the verification is arbitrary as longas it is three or more wherein comparison can be conducted betweenadjacent pixels at the same field and pixels at different fields.Moreover, the position of the pixel to be verified may be any of all ofthe pixels inside the video picture. Otherwise, a sample point maybeinvestigated such that the above-described verification is conducted ata specific position or an arbitrary position inside one block composedof, for example, five pixels in the vertical direction and n pixels inthe horizontal direction. Alternatively, utterly arbitrary points may bespot-checked at random.

Subsequently, explanation will be made on the processing of thedownscaled feature plane creation section 53 illustrated in FIG. 11,i.e., the processing of creating a downscaled plane in consideration ofthe feature of the video picture based on an original video picture inreference to FIG. 14. First, the original video picture is divided intosmall blocks, each of which is expressed by a representative value.According to the present invention, the standard deviation of the pixelvalues per small block is used as the representative value. An averagevalue or a median value may be used as the representative value. Aluminance component of the pixel may be used as the pixel value at thetime of the calculation; or, other components or an average thereof maybe used. Furthermore, the size of the small block may be arbitrarilyset. Assuming that the small block is composed of ph pixels in thehorizontal direction multiplied by pv pixels in the vertical direction,the downscaled feature plane is composed of H/ph pixels in thehorizontal direction multiplied by V/pv pixels in the vertical directionwith respect to the size of the original video picture (H pixels in thehorizontal direction multiplied by V pixels in the vertical direction),so that the number of samples becomes 1/(ph×pv) with respect to thenumber of pixels of the original video picture. The downscaled planehaving the standard deviation of the small block as the representativevalue is the downscaled feature plane information 54.

Next, description will be given below of the processing by the simplemotion estimation section 55 illustrated in FIG. 11. The simple motionestimation section 55 performs the motion estimation processing betweenthe two downscaled feature planes based on the downscaled feature planeinformation 54 created by the downscaled feature plane creation section53. A timewise distance between a reference plane and a target plane tobe subjected to the simple motion estimation is an arbitrarily fixedvalue. Motion estimation by block matching or the like can be used inthe motion estimating method. In this case, the block can take anarbitrary natural number for both of horizontal and vertical sizes onthe downscaled feature plane. Consequently, the motion estimation perblock can be carried out by using, as one block, one sample at theminimum or the entirety of one downscaled feature plane at the maximum.

Referring to FIG. 15, explanation will be made below. An upper leftcoordinate of the set block is designated by (k,l); an element on adownscaled feature plane 1, c(k,l); and an element on a downscaledfeature plane 2, r(k,l). Reference character N represents the size ofthe block in the horizontal direction; and M, the size of the block inthe vertical direction. The estimation range falls within ±sh in thehorizontal direction and ±sv in the vertical direction. An averagemotion compensatory prediction error E(k,l) of one element in thissimple motion estimation is determined based on the minimum error withinthe estimation range according to the following equations (5) and (6):E(k,l)=Min(Err(k,l,h,v))  (5)here,

$\begin{matrix}{{{{Err}\left( {k,l,h,v} \right)} = {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{{c\left( {{k + m},{l + n}} \right)} - {r\left( {\left( {k + m + h} \right),\left( {l + n + v} \right)} \right)}}}}}}\left( {{{- {sh}} \leqq h \leqq {sh}},{{- {sv}} \leqq v \leqq {sv}}} \right)} & (6)\end{matrix}$

As to the prediction error E(k,l), square root processing may beperformed after determination of a square error, or an absolutedifference may used. The prediction error E(k,l) obtained by the simplemotion estimation processing is determined with respect to all of theblocks on the downscaled feature plane 1, thereby obtaining the sum Esumon the downscaled feature plane 1. The sum Esum is an index indicatingthe magnitude of a variation between two video pictures. The sum Esum asthe image variance information 56 is output to the frame/field structuredecision section 57.

Next, the frame/field structure decision section 57 illustrated in FIG.11 judges whether or not the image variance information 56 per inputframe exceeds a threshold value. If the image variance information 56per input frame is the threshold value or more, the frame/fieldstructure decision section 57 decides the field structure; to thecontrary, if the image variance information 56 per input frame is lessthan the threshold value, the frame/field structure decision section 57decides the frame structure. Thereafter, the frame/field structuredecision section 57 outputs the decision result as the picture structureinformation 58 to the video coding section 59.

The video coding section 59 illustrated in FIG. 11 performs thecompression coding of the video signal to be input by the use of thepicture structure designated by the picture structure information 58output from the frame/field structure decision section 57, and then,outputs the coded data 16. Specifically, the video coding section 59switches, for example, the operations of the motion compensator 10, thefirst variable length encoder 5 and the second variable length encoder14 illustrated in FIG. 1 to the system adaptive to coding in theframe/field structure according to the picture structure.

FIG. 16 is a block diagram illustrating the configuration of a sixthembodiment according to the present invention. The same referencenumerals as those in FIG. 11 designate like or corresponding constituentelements. In the present embodiment, a video coding apparatus comprisesa downscaled feature plane creation section 53, a simple motionestimation section 55 and a frame/field structure decision section 57.The present embodiment is characterized in that the frame/fieldstructure decision section 57 selects coding by a field structure ifimage variance information 56 obtained by simple motion estimationprocessing in the simple motion estimation section 55 exceeds a certainthreshold value; in the meantime, it selects coding by a frame structureif the image variance information 56 is less than the threshold value.There is a difference between the fifth embodiment illustrated in FIG.11 and the present embodiment in that in the former embodiment thedownscaled feature plane creation section 53 creates the downscaledfeature plane in the case of the interlaced video picture, while in thelatter the downscaled feature plane creation section 53 creates thedownscaled feature plane also in the case of a non-interlaced videopicture.

FIG. 17 is a block diagram illustrating the configuration of a seventhembodiment according to the present invention. The same referencenumerals as those in FIG. 11 designate like or corresponding constituentelements. In the present embodiment, a video coding apparatus comprisesan interlaced/non-interlaced video discriminant section 51 and aframe/field structure decision section 57. The interlaced/non-interlacedvideo discriminant section 51 discriminates whether or not an inputvideo picture is an interlaced video picture. The present embodiment ischaracterized in that the frame/field structure decision section 57selects coding by a field structure in the case where the input videopicture is an interlaced video picture; to the contrary, it selectscoding by a frame structure in the case where the input video picture isa non-interlaced video picture.

FIG. 18 is a block diagram illustrating the configuration of an eighthembodiment according to the present invention. The same referencenumerals as those in FIG. 11 designate like or corresponding constituentelements. In the present embodiment, a video coding apparatus comprisesan interlaced/non-interlaced video discriminant section 51, aninterlaced/non-interlaced video switch section 60, a downscaled featureplane creation section 53, a simple motion estimation section 55 and aframe/field structure decision section 57. The interlaced/non-interlacedvideo discriminant section 51 discriminates whether one video pictureinput first or a plurality of video pictures are interlaced ornon-interlaced video pictures. Based on the discrimination, theinterlaced/non-interlaced switch section 60 switches “0” and “1”. As forvideo pictures input hereafter, the interlaced/non-interlaced videodiscriminant section 51 does not perform discrimination of interlaced ornon-interlaced video pictures. The present embodiment is different inthe above-described point from the fifth embodiment illustrated in FIG.11.

FIG. 19 is a block diagram illustrating the configuration of a ninthembodiment according to the present invention. The same referencenumerals as those in FIG. 11 designate like or corresponding constituentelements. In the present embodiment, a video coding apparatus comprisesan interlaced/non-interlaced video discriminant section 51, aninterlaced/non-interlaced video switch section 60 and a frame/fieldstructure decision section 57. The interlaced/non-interlaced videodiscriminant section 51 discriminates whether one video picture inputfirst or a plurality of video pictures are interlaced or non-interlacedvideo pictures. Based on the discrimination, theinterlaced/non-interlaced switch section 60 switches “0” and “1”. As forvideo pictures input hereafter, the interlaced/non-interlaced videodiscriminant section 51 does not perform discrimination of interlaced ornon-interlaced video pictures. The present embodiment is different inthe above-described point from the seventh embodiment illustrated inFIG. 17.

As is obvious from the above description, although the video picturehaving an improved coding efficiency is limited in the conventionalcoding by the fixed picture structure, since the coding is selecteddepending on the picture structure according to the feature or variationof the input video picture according to the present invention, a highcoding efficiency can be maintained even if a video picture having anyfeature is input or the feature of the video picture is varied on theway.

Furthermore, the video coding simulation is conducted by using the MPEG2video coding system as the video coding system in which the motioncompensatory prediction coding can be carried out by either the framestructure or the field structure. As a result, the quality of the videopicture can be improved by about 0.4 dB to 1.0 dB of PSNR according tothe present invention in comparison with the case of the fixation in theframe structure under the condition of the compression coding at acoding rate of 4 Mbit/s.

1. A video coding apparatus for coding a video picture by either a fieldstructure or a frame structure, the video coding apparatus comprising:means for discriminating whether each of sequentially input videopictures is an interlaced video picture or a non-interlaced videopicture; and means for coding by the field structure in response todiscriminating that the video picture is an interlaced video picture andfor coding by the frame structure in response to discriminating that thevideo picture is a non-interlaced video picture, wherein in order todiscriminate whether the input video picture is an interlaced videopicture or a non-interlaced video picture, the spatial correlation ofpixels continuous in a vertical direction at an arbitrary positioninside the video picture is measured, so that the video picture isdiscriminated to be an interlaced video picture if the correlationbetween the same fields is higher than the correlation between differentfields.
 2. A video coding apparatus according to claim 1, wherein thecoding by the field structure is selected in the case where the numberof pixels satisfying the conditions expressed by inequalities (1) and(2) below exceeds a predetermined rate of the number of pixelssatisfying the inequality (1) in measuring the spatial correlation ofthe pixels continuous in the vertical direction:Max(d(0,−2),d(0,2),d(−1,1))<threshold value  (1)(Max(d(0,−2),d(0,2),d(−1,1))+offset)<Min(d(0,−1),d(0,1))  (2) wherein, aand b represent pixel position in the vertical direction, d (a,b)represents an absolute difference between a and b.
 3. A video codingapparatus for coding a video picture by either a field structure or aframe structure, the video coding apparatus comprising: means forcalculating a correlation between two video pictures with a timewiseinterval with respect to sequentially input video pictures; and meansfor deciding whether the coding is carried out by either a fieldstructure or a frame structure based on the correlation, wherein thecoding by the frame structure is carried out in the case of thecorrelation being higher than a predetermined value while the coding bythe field structure is carried out in the case of the correlation beinglower than the predetermined value, wherein the means for calculatingthe correlation between the two video pictures comprises: means forcreating a downscaled plane in consideration of features of sequentiallyinput video pictures; and means for performing simple motion estimationprocessing on the downscaled plane, and wherein the coding by the fieldstructure is selected in the case where a motion compensation predictionerror obtained by the simple motion estimation processing is larger thana predetermined value.
 4. A video coding apparatus according to claim 3,wherein the means for creating the downscaled plane in consideration ofthe feature of the video picture divides the video picture into smallblocks and calculates a deviation per divided small block, the deviationbeing an element of the downscaled plane.
 5. A video coding apparatusaccording to claim 3, further comprising means for discriminatingwhether the input video picture is an interlaced video picture or anon-interlaced video picture, wherein a video picture variance isanalyzed, so that the coding by the field/frame structure is selected bydetecting the correlation between the two video pictures with respect toonly the video pictures which are discriminated to be interlaced videopictures, while the coding by the frame structure is selected withrespect to the video pictures which are not discriminated to beinterlaced video pictures.
 6. A video coding apparatus according toclaim 5, further comprising means for switching and setting theinterlaced/non-interlaced video pictures, wherein it is discriminatedwhether one video picture input first or a plurality of video picturesare interlaced video pictures or non-interlaced video pictures, so thatthe means for switching and setting the interlaced/non-interlaced videopictures is set based on the discrimination result.
 7. A video codingapparatus according to claim 3, further comprising means fordiscriminating whether the input video picture is an interlaced videopicture or a non-interlaced video picture, wherein a video picturevariance is analyzed, so that the coding by the field/frame structure isselected by detecting the correlation between the two video pictureswith respect to only the video pictures which are discriminated to beinterlaced video pictures, while the coding by the frame structure isselected with respect to the video pictures which are not discriminatedto be interlaced video pictures.
 8. A video coding apparatus accordingto claim 7, further comprising means for switching and setting theinterlaced/non-interlaced video pictures, wherein it is discriminatedwhether one video picture input first or a plurality of video picturesare interlaced video pictures or non-interlaced video pictures, so thatthe means for switching and setting the interlaced/non-interlaced videopictures is set based on the discrimination result.